Person Localization in a Wearable Camera Platform towards Assistive Technology for Social Interactions

Social interactions are a vital aspect of everyone’s daily living. Individuals with visual impairments are at a loss when it comes to social interactions as majority (nearly 65%) of these interactions happen through visual non-verbal cues. Recently, efforts have been made towards the development of an assistive technology, called the Social Interaction Assistant (SIA)[1], which enables access to non-verbal cues for individuals who are blind or visually impaired. Along with self report feedback about their own social interactions, behavioral psychology studies indicate that individuals with visual impairment will benefit in their social learning and social feedback by gaining access to non-verbal cues of their interaction partners. As part of this larger SIA project, in this paper, we discuss the importance of person localization while building a human-centric assistive technology which addresses the essential needs of the visually impaired users. We describe the challenges that arise when a wearable camera platform is used as a sensor for picking up non-verbal social cues, especially the problem of person localization in a real-world application. Finally, we present a computer vision based algorithm adapted to handle the various challenges associated with the problem of person localization in videos and demonstrate its performance on three examplar video sequences.